Fundamental subspaces

Lecture 19

Author

Affiliation

Minjae Park

Auburn University
MATH 2660 - Spring 2026

Published

February 23, 2026

Recap & Motivation

Least Squares Approximation

Often the system \(A\vec{x} = \vec{b}\) has no exact solution.
In this case, we seek \(\vec{x}\) that gives the best possible approximation to \(\vec{b}\).
Define the error vector by \(\vec{e} = A\vec{x} - \vec{b}\).
We choose \(\vec{x}\) to minimize the error length \(\|\vec{e}\|\).
The error is minimized precisely when \(\vec{e}\) is orthogonal to the range of \(A\).
This orthogonality condition leads to the normal equations \[ A^T A \vec{x} = A^T \vec{b}. \]

Illustration

Geometric Interpretation

Let \(A\) be an \(n \times m\) matrix.
Its range (or column space) is \[ R(A) = \{A\vec{x} : \vec{x} \in \mathbb{R}^m\} \subseteq \mathbb{R}^n. \]
If \(\vec{b} \notin R(A)\), then the system \(A\vec{x} = \vec{b}\) has no exact solution.
The range \(R(A)\) equals the span of the columns of \(A\), a subspace of \(\mathbb{R}^n\).
The least squares solution produces \(A\vec{x}\), the orthogonal projection of \(\vec{b}\) onto \(R(A)\).
More broadly, every matrix has four fundamental subspaces, one of which is \(R(A)\).

Fundamental Subspaces

Column Space of a Matrix

Let \(A = [\vec{u}_1 \ \cdots \ \vec{u}_k]\) be an \(n \times k\) matrix with \(\vec{u}_i \in \mathbb{R}^n\).
View \(A\) as a linear transformation from \(\mathbb{R}^k\) to \(\mathbb{R}^n\).
Each standard basis vector \(\vec{e}_i \in \mathbb{R}^k\) is mapped to the column vector \(\vec{u}_i\).
Thus the standard basis \(\{\vec{e}_1, \dots, \vec{e}_k\}\) is transformed into the spanning set \(\{\vec{u}_1, \dots, \vec{u}_k\}\).
Although there are \(k\) columns, the dimension of the space they span may be smaller (if the columns are linearly dependent).
The column space of \(A\) is \[ C(A) = \mathop{\mathrm{span}}\{\vec{u}_1, \dots, \vec{u}_k\} \subseteq \mathbb{R}^n. \]
Equivalently, the column space is the range of \(A\): \(R(A) = \{A\vec{x} : \vec{x} \in \mathbb{R}^k\}\).

Rank of a Matrix

Ideally, if \(S = \{\vec{u}_1, \dots, \vec{u}_k\}\) is linearly independent, then \(C(A)\) is \(k\)-dimensional.
If the columns are linearly dependent, then \(C(A)\) has dimension less than \(k\).
The (column) rank of \(A\) is defined as \[ \mathop{\mathrm{rank}}(A) = \dim(C(A)). \]
Rank measures the maximum number of linearly independent columns (i.e., the number of vectors after every redundancy removed).

Pivot Columns

Let \(A = [\vec{u}_1 \ \cdots \ \vec{u}_k]\) be an \(n\times k\) matrix with \(\vec{u}_i \in \mathbb{R}^n\).
In the context of linear systems, \(A\) represents \(n\) equations in \(k\) variables.
When solving \(A\vec{x}=\vec{0}\), the columns containing pivots (the first nonzero entry in each nonzero row of the RREF) correspond to non-free variables.
These non-free variables are determined once the free variables (parameters) are chosen.
The columns of the original matrix \(A\) corresponding to pivot columns in \(\operatorname{RREF}(A)\) are called the pivot columns of \(A\).
These pivot columns are linearly independent and form a basis of \(C(A)\).

Example

Let \[ A= \begin{pmatrix} 1 & 0 & 1 & 1 \\ 0 & 0 & 1 & 3 \\ 1 & 0 & 2 & 4 \end{pmatrix}. \]
Row reduction gives \[ \operatorname{RREF}(A)= \begin{pmatrix} 1 & 0 & 0 & -2 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 \end{pmatrix}. \]
Give a basis for \(C(A)\) and determine \(\mathop{\mathrm{rank}}(A)\).
Answer: \(\{\langle 1,0,1 \rangle,\ \langle 1,1,2 \rangle\}\) and \(2\).

Row Space of a Matrix

Each row of a matrix can also be viewed as a vector (e.g. in the context of dot products).
To follow our column-vector convention, we may take the transpose so that rows become columns.
The row space of \(A\) is the vector space spanned by the rows of \(A\), which is the same as \(C(A^T)\).
The (row) rank of \(A\) is defined as
\[\mathop{\mathrm{rank}}(A^T) = \dim(C(A^T)).\]

Theorem: Column Rank = Row Rank

The column rank and the row rank of a matrix are always equal. We call this common value the rank of \(A\), denoted by \(\mathop{\mathrm{rank}}(A)\).
Idea: Both the column rank and the row rank equal the number of pivots in \(\operatorname{RREF}(A)\).
(For a formal proof, see the 🔗link.)
Caution: In general, \(C(A)\ne C(A^T)\); only their dimensions are equal.

Example

Let \[ A = \begin{pmatrix} 1 & -2 & 0 \\ 2 & -4 & 1 \end{pmatrix}. \]
The column vectors form \(S=\{\langle 1,2 \rangle,\ \langle -2,-4 \rangle,\ \langle 0,1 \rangle\}\).
The first two vectors are linearly dependent, so \(\dim(\mathop{\mathrm{span}}(S))=2\).
Hence \(C(A)=\mathop{\mathrm{span}}(S)=\mathbb R^2\) and \(\mathop{\mathrm{rank}}(A)=2\).
The row vectors \(\langle 1,-2,0 \rangle\) and \(\langle 2,-4,1 \rangle\) are linearly independent.
Thus \(C(A^T)=\mathop{\mathrm{span}}(\langle 1,-2,0 \rangle,\ \langle 2,-4,1 \rangle)\) is a plane in \(\mathbb R^3\).
Still, \(\mathop{\mathrm{rank}}(A^T)=2=\mathop{\mathrm{rank}}(A)\), although \(C(A)\ne C(A^T)\).

Exercise

Let \[ A= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 1 & 1 & 0 \end{pmatrix}. \]
Determine bases for \(C(A)\) and \(C(A^T)\), and find \(\mathop{\mathrm{rank}}(A)\).
Only answer \(\mathop{\mathrm{rank}}(A)\) on iClicker.

Scan the QR code or go to join.iclicker.com/MBNJ.

Nullspace of a Matrix

Another fundamental subspace associated with \(A\) arises from solving linear equations.
The nullspace of \(A\) is
\[ N(A) = \{\vec{x} \in \mathbb{R}^m : A\vec{x} = \vec{0}\}. \]
Thus, \(N(A)\) is the solution space of the homogeneous system \(A\vec{x} = \vec{0}\).
Exercise: Verify that \(N(A)\) is a vector space.
The nullity of \(A\) is defined as
\[\mathop{\mathrm{null}}(A) = \dim(N(A)).\]
The space \(N(A^T)\) is called the left nullspace of \(A\), and \(\mathop{\mathrm{null}}(A^T) = \dim(N(A^T))\) is called the left nullity of \(A\).

Rank and Nullity from RREF

We use RREF to determine whether a set of vectors is linearly independent.
RREF also reveals how many linearly independent columns there are.
The number of pivots in the RREF of \(A\) equals the rank of \(A\).
The number of columns minus the number of pivots equals the number of free variables.
This number of free variables is exactly \(\mathop{\mathrm{null}}(A)\).

Exercise

Let \[ A= \begin{pmatrix} 1 & -1 & 0 \\ 2 & -2 & 1 \end{pmatrix}. \]
Compute \(\operatorname{RREF}(A)\).
Indicate the locations of the pivots (the first nonzero entry in each nonzero row).
Find \(\mathop{\mathrm{rank}}(A)\) and \(\mathop{\mathrm{null}}(A)\).

Exercise

Let \[ A= \begin{pmatrix} 1 & 2 & 1 \\ 2 & 4 & 0 \\ 1 & 2 & 3 \\ 3 & 6 & 1 \end{pmatrix}. \]
Compute \(\operatorname{RREF}(A)\).
Find \(\mathop{\mathrm{rank}}(A)\) and \(\mathop{\mathrm{null}}(A)\).

Rank–Nullity Theorem

Let \(A\) be an \(n\times m\) matrix.
Then
\[ \mathop{\mathrm{rank}}(A) + \mathop{\mathrm{null}}(A) = m. \]
Similarly,
\[ \mathop{\mathrm{rank}}(A^T) + \mathop{\mathrm{null}}(A^T) = \mathop{\mathrm{rank}}(A) + \mathop{\mathrm{null}}(A^T) = n. \]

Summary

Let \(A\) be an \(n\times m\) matrix. There are four fundamental subspaces.
Column space \(C(A) \subseteq \mathbb{R}^n\) has dimension \(\mathop{\mathrm{rank}}(A)\).
Row space \(C(A^T) \subseteq \mathbb{R}^m\) has dimension \(\mathop{\mathrm{rank}}(A)\).
Nullspace \(N(A) \subseteq \mathbb{R}^m\) has dimension \(\mathop{\mathrm{null}}(A)\).
Left nullspace \(N(A^T) \subseteq \mathbb{R}^n\) has dimension \(\mathop{\mathrm{null}}(A^T)\).
Rank–nullity theorem:
\[\mathop{\mathrm{rank}}(A) + \mathop{\mathrm{null}}(A) = m,\qquad \mathop{\mathrm{rank}}(A) + \mathop{\mathrm{null}}(A^T) = n.\]

Implications

Let \(A\) be an \(n\times m\) matrix. Then \[ \mathop{\mathrm{rank}}(A)\le \min(n,m), \] since \(\mathop{\mathrm{null}}(A)\ge 0\) and \(\mathop{\mathrm{null}}(A^T)\ge 0\). (Equivalently, the number of linearly independent columns or rows cannot exceed the total number of columns or rows.)
If \(\mathop{\mathrm{rank}}(A)=m\) (the number of columns), then the columns of \(A\) are linearly independent. If \(\mathop{\mathrm{rank}}(A)=n\) (the number of rows), then the rows of \(A\) are linearly independent.
Conceptually, the rank measures how much essential information is encoded in the matrix. Even if a matrix is very large, if its rank is \(r\) (with \(r\) small), then its behavior is essentially governed by \(r\) independent directions — it acts like an \(r\)-dimensional object inside a much larger space.